<Paper Log> SmartPlayer： User-Centric Video Fast-Forwarding

• Should read again when planning kineto project.

• As an experience when watching Video, it realizes something like the scenery while driving.

  • Based on metaphor.
  • (blu3mo) Now that I think about it, there is a similarity between this metaphor and Axes for immersion, where time and space merge.

• Can freely adjust the speed while watching.#fast-forward#slow-motion

• When there is no time, users have to skip videos based on intuition with existing interfaces.

• Let’s do it automatically.

• Supports skimming through videos.

• There are two methods for video summarization:

  • still-image abstraction
    • Arranging images from parts of the video.
  • video skimming
    • Cutting and connecting parts of the video?

• When using video skimming to skip unfamiliar videos, users become anxious about missing information.

• Features:

  • Can freely control the playback speed based on the complexity of the content at that time.
  • Like using GPS navigation while driving, important points are indicated on the playback bar (POI).
  • There is also a mode to automatically adjust the speed, like autonomous driving.
    • Adjusts the speed while observing the movement of video frames and the semantic aspects of the video using machine learning.
      • This is achieved through three layers.
      • Motion Layer
        • Observes optical flow using the Lucas-Kanade method and calculates based on its values.
      • Semantic Layer
        • For this study, it was manually set, but there are also automatic models available for different types of videos.
        • For example, there are models that extract semantic points in wedding videos.
        • The essence of this research is not here, so it was manually set.
      • Personalization Layer
        • Adjusts the learning based on user interactions.
    • Personalize it.

• Previous studies included various video interactions.

• Also included methods for video summarization.

  • Could be useful for organizing notes.

• Thoughts:

  • Are aiming for something amazing?
  • The approach to summarizing videos is useful.
  • Want to think about doing this in real-time (without hindsight).
  • Can machine learning be used by taking advantage of multiple people attending a lecture at the same time?
  • This feedback from users is helpful.
    • “I skipped through political news because I wasn’t interested, but I didn’t skip 10 seconds. Because I wanted to have a rough understanding.”
    • In school, I want to confirm whether I understand all the content while skipping what I already know = fast-forward, not skipping 10 seconds.
  • It may be possible to improve the experience if users have prior knowledge about the video.
    • For example, when someone who understands the flow of a baseball game watches, the seek bar could provide information about the game so that they can manually skip through the video.

• There is a lot of knowledge about time manipulation interfaces.

• Which papers to read next?

  • These two seem relevant to video analysis in unique environments like kineto:
    • [[ Explicit Semantic Events Detection and Development of Realistic Applications for Broadcasting Baseball Videos]]
    • [[ Semantic Analysis for Automatic Event Recognition and Segmentation of Wedding Ceremony Videos]]
  • Papers related to video summarization:
    • [[ An Extended Framework for Adaptive Playback-Based Video Summarization]]

https://www.researchgate.net/publication/221518075_Smartplayer_User-centric_video_fast-forwarding #kineto